Detecting Missing Annotation Disagreement using Eye Gaze Information
نویسندگان
چکیده
This paper discusses the detection of missing annotation disagreements (MADs), in which an annotator misses annotating an annotation instance while her counterpart correctly annotates it. We employ annotator eye gaze as a clue for detecting this type of disagreement together with linguistic information. More precisely, we extract highly frequent gaze patterns from the pre-extracted gaze sequences related to the annotation target, and then use the gaze patterns as features for detecting the MADs. Through the empirical evaluation using the data set collected in our previous study, we investigated the effectiveness of each type of information. The results showed that both eye gaze and linguistic information contributed to improving performance of our MAD detection model compared with the baseline model. Furthermore, our additional investigation revealed that some specific gaze patterns could be a good indicator for detecting the MADs.
منابع مشابه
Parameter estimation of Japanese predicate argument structure analysis model using eye gaze information
In this paper, we propose utilising eye gaze information for estimating parameters of a Japanese predicate argument structure (PAS) analysis model. We employ not only linguistic information in the text, but also the information of annotator eye gaze during their annotation process. We hypothesise that annotator’s frequent looks at certain candidates imply their plausibility of being the argumen...
متن کاملSemi-automatic annotation of eye-tracking recordings in terms of human torso, face and hands
Research on interactive communication increasingly focuses on the role of eye gaze as an important signal in interaction management, reference and grounding (see [1] for an overview). Interlocutors may use eye gaze as a means to take, hold or give the floor in conversation (turn management), to refer to objects or persons in the conversational space (gaze cueing) or to give and elicit feedback ...
متن کاملReading behavior predicts syntactic categories
It is well-known that readers are less likely to fixate their gaze on closed class syntactic categories such as prepositions and pronouns. This paper investigates to what extent the syntactic category of a word in context can be predicted from gaze features obtained using eye-tracking equipment. If syntax can be reliably predicted from eye movements of readers, it can speed up linguistic annota...
متن کاملInvestigation of annotator's behaviour using eye-tracking data
This paper presents an analysis of an annotator’s behaviour during her/his annotation process for eliciting useful information for natural language processing (NLP) tasks. Text annotation is essential for machine learning-based NLP where annotated texts are used for both training and evaluating supervised systems. Since an annotator’s behaviour during annotation can be seen as reflecting her/hi...
متن کاملEye contact does not facilitate detection in children with autism.
Eye contact is crucial in achieving social communication. Deviant patterns of eye contact behavior are found in individuals with autism, who suffer from severe social and communicative deficits. This study used a visual oddball paradigm to investigate whether children with high functioning autism have difficulty in detecting mutual gaze under experimental conditions. The results revealed that c...
متن کامل